WWC Intervention Report 

U.S. DEPARTMENT OF EDUCATION 

What Works Clear inghouse™ 

# # 

1 ^ INSTITUTE of 

■ W 4# EDUCATION SCIENCES 

Beginning Reading 

Updated February 2015 

Houghton Mifflin Reading® 


No studies of Houghton Mifflin Reading ® that fall within the scope of the Beginning Reading review 
protocol meet What Works Clearinghouse (WWC) group design standards. Because no studies meet 
WWC group design standards at this time, the WWC is unable to draw any conclusions based on 


research about the effectiveness or ineffectiveness of Houghton Mifflin Reading ® on beginning 
readers in grades K-3. Research that meets WWC standards is needed to determine the effectiveness 
or ineffectiveness of this intervention. 

Program Description 1 

Houghton Mifflin Reading © is a reading program designed for grades K-6. The program provides step-by-step instruc- 
tion in reading using Big Books (fiction and nonfiction literature), anthologies, Read Aloud books, and audio com- 
pact discs. The product is designed to be used as a full-year curriculum program with instruction on developing oral 
language, comprehension, phonemic awareness, decoding skills (phonics, analogy, context, and word recognition), 
fluency, reading comprehension, writing, spelling, and grammar. Instruction is organized by a set of themes (10 for 
grades K-1 , and six for grades 2-6) with selected Big Books and other classroom activities to highlight each theme. 
Themes for grade 1 , for example, include “Family and Friends,” “All Together Now,” and “Let’s Look Around!” This 
review of the program for the Beginning Reading topic area focuses on beginning readers in grades K-3. 

Research 2 

The WWC identified 10 studies of Houghton Mifflin Reading © for beginning readers in grades K-3 that were 
published or released between 1983 and 2014. 

Five studies are within the scope of the Beginning Reading review protocol but do not meet WWC group design 
standards. 

• Three studies used a quasi-experimental design where measures of effectiveness could not be attributed solely 
to the intervention. 

• Two studies used a quasi-experimental design and did not establish that the intervention group was comparable 
to the comparison group prior to the start of the intervention. 

Five studies are out of the scope of the Beginning Reading review protocol for reasons other than study design. 

• Three studies did not use a sample aligned with the protocol. 

• Two studies did not examine the effectiveness of the intervention in a way that falls within the scope of 
the protocol. 
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Endnotes 

1 The descriptive information for this program was obtained from a publicly available source: the program’s website (http://www. 
hmhco.com, downloaded August 2014). The WWC requests developers to review the program description sections for accuracy from 
their perspective. The program description was provided to the developer in August 2014; however, the WWC received no response. 
Further verification of the accuracy of the descriptive information for this program is beyond the scope of this review. 

2 The literature search reflects documents publicly available by April 2014. This report has been updated to include reviews of four 
studies that have been released since 2008. (The previous report was released in September 2008.) Of the additional studies, two 
were not within the scope of the protocol, and two were within the scope of the protocol but did not meet WWC group design stan- 
dards. A complete list and disposition of all studies reviewed are provided in the references. The studies in this report were reviewed 
using the Standards from the WWC Procedures and Standards Handbook (version 3.0), along with those described in the Beginning 
Reading review protocol (version 3.0). The evidence presented in this report is based on available research. Findings and conclusions 
may change as new research becomes available. 

Recommended Citation 
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Houghton Mifflin Reading® Updated February 201 5 


Page 4 


WWC Intervention Report 


Glossary of Terms 

Attrition 

Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 

Single-case design 
Standard deviation 


Statistical significance 


Substantively important 


Attrition occurs when an outcome variable is not available for all participants initially assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review and inclusion in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent of 
evidence levels are given in the WWC Procedures and Standards Handbook (version 3.0). 

Along a percentile distribution of individuals, the improvement index represents the gain 
or loss of the average individual due to the intervention. As the average individual starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which study participants are 
assigned to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which eligible study participants are 
randomly assigned to intervention and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the research 
design and the magnitude, statistical significance, and consistency in findings. The criteria for the 
ratings of effectiveness are given in the WWC Procedures and Standards Handbook (version 3.0). 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample tend to be spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < .05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 
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